"Getting in" revisited: an analysis of manuscript characteristics, reviewers' ratings, and acceptance of manuscripts in Psychological Bulletin.

Getting published in the premier journals in psychology can be a daunting task, especially for the novice. Young scholars or those with limited experience publishing often ask editors what factors affect the probability of a manuscript being accepted and what they should do to enhance their chances of getting published. Of course, many factors affect whether a manuscript is accepted, including who the acting editor and reviewers happen to be and the quality and orientation of the journal to which a paper is submitted. Nonetheless, it is highly likely that certain characteristics of manuscripts contribute in a systematic manner to reviewers’ evaluations of them and, ultimately, to a final editorial decision. At the close of Robert J. Sternberg’s editorship of Psychological Bulletin, Sternberg, Hojjat, Brigockas, and Grigorenko (1997) published an analysis of reviewers’ ratings and their relation with the final editorial decisions for manuscripts handled by the editor and one of the associate editors between January 1, 1993, and February 29, 1996. In their analyses, they considered each item on the reviewer rating form as addressing an analytical, creative, or practical aspect of the manuscript. The analytically oriented items concerned the extent to which “the authors completely and correctly analyzed, evaluated, and reflected on the extant literature reviewed” (p. 321). These items included completeness of coverage, balance and fairness in the coverage of alternative views, and accuracy of information. Thus, to score high on this dimension, authors needed to accurately determine what articles should be reviewed, analyze their views and findings in a balanced manner, and provide an accurate summary of the issue in question. Sternberg et al. (1997) included among the creativity items those that assessed whether the article made a novel contribution to the field. The items that tapped this dimension included scientific importance of the topic, theoretical orientation of manuscript, and contribution to the field reviewed. A creative article, then, was viewed as one that involved “taste and judgment in topic selection; . . . the formation of a new theory or application of an existing theory to diverse scientific findings; . . . and a substantial incremental contribution to a field at the time of publication” (p. 321). The items Sternberg et al. (1997) labeled as practically oriented concerned the extent to which manuscripts reflected authors’ tacit knowledge of the features of a manuscript that make it publishable in a particular journal. For Psychological Bulletin, these included “generalized writing skills needed to communicate effectively and persuasively to a diverse audience” (p. 321). The items considered as tapping this dimension were scope of review; appropriateness of manuscript for journal; interest for a broad audience of psychologists; existence and clarity of a take-home message; existence of and persuasiveness in arguing for a well-articulated point of view; organization of manuscript; and clarity, coherence, and conciseness of prose. Although Sternberg et al. (1997) initially classified their rating items into the aforementioned three classes, an exploratory factor analysis of reviewers’ item ratings for initial manuscript reviews (not revisions) suggested two factors, which they described as the overall appropriateness of manuscripts for any scientific journal and the overall appropriateness of manuscripts specifically for Psychological Bulletin. In addition, they found that all items had statistically significant polyserial correlations with the final editorial decision regarding acceptance versus rejection of a manuscript. The items most strongly correlated with the final editorial decision (in order of strength) were the reviewer’s recommendation regarding acceptance/rejection and contribution to the field reviewed, followed by appropriateness of the manuscript for the journal, existence of and persuasiveness in arguing for a well-articulated point of view, completeness of coverage, and theoretical orientation of the manuscript. Our goal in this article is to reexamine manuscript characteristics that affect reviewers’ recommendations regarding acceptance versus rejection of manuscripts and the acting editor’s final decision. Early in Nancy Eisenberg’s editorship, she devised a rating sheet for reviewers that was used subsequently by all associate editors. Many items on the full 13-item scale used under her editorship paralleled those used on Robert J. Sternberg’s rating form with minor wording changes, although multiple items were replaced or rewritten to capture more specific technical aspects of the manuscript (e.g., quality of the analyses [if relevant]). Building on Sternberg et al.’s (1997) analysis and considering the changes in the rating form, we expected to find three dimensions of manuscript characteristics to be useful for predicting publication in Editor’s Note. Daniel Reisberg served as the action editor for this article.—NE